General Convergence Analysis of Stochastic First-Order Methods for Composite Optimization

In this paper, we consider stochastic composite convex optimization problems with the objective function satisfying a bounded gradient condition, or without quadratic functional growth property. These models include most well-known classes of functions analyzed in literature: nonsmooth Lipschitz and composition (potentially) smooth function, strong convexity. Based on flexibility offered by our model, several variants first-order methods, such as proximal point algorithms. Usually, convergence theory for these methods has been derived simple restrictive assumptions, rates are general sublinear hold only specific decreasing stepsizes. Hence, analyze constant variable stepsize under assumptions covering large class functions. For stepsize, show that can achieve linear rate up to proportional some condition even pure convergence. Moreover, when is chosen derive methods. Finally, mapping Moreau smoothing introduced present paper lead intuitive proofs.